High-Dimensional Learning Under Approximate Sparsity with Applications to Nonsmooth Estimation and Regularized Neural Networks

In “High-Dimensional Learning Under Approximate Sparsity with Applications to Nonsmooth Estimation and Regularized Neural Networks,” Liu, Ye, Lee study a model fitting problem where there are much fewer data than dimensions. Of their particular focus the scenarios commonly imposed sparsity assumption is relaxed, usual condition of restricted strong convexity absent. The results show that generalization performance can still be ensured in such settings, even if dimensions grow exponentially. authors further sample complexities high-dimensional nonsmooth estimation neural networks. Particularly for latter, it shown that, explicit regularization, network provably generalizable, size only poly-logarithmic number parameters.